<html>
<head>
<title>OSPFv2 Testsuite - Documentation</title>
<link rel="stylesheet" href="testsuite.css" type="text/css">
</head>
<body>
<br><hr>
<div align="center"> 
<a href="http://ssfnet.d-jacob.net">Home</a> | 
<a href="index.html">Up</a> | 
<a href="ospfv2.html">Previous</a> | 
<a href="design.html">Next</a>
</div>
<hr><br>
<h1>Test Suite Selection</h1>

The OSI framework for conformance testing makes no assumptions on how the test cases 
should be developed. Formal methods can be used as well as informal ones. Since the 
applicability of methods for a particular protocol may vary, the decision for a method 
must be made separately for each protocol. There are different criteria for the 
selection of a test selection method:<p>

<ol>
<li> Correctness of the resulting test suite.
<li> Coverage of the test suite.
<li> Relevance and acceptance in practice.
<li> Ease of implementation in SSFNet.
<li> Applicability of the test suite during development.
<li> Availability of the test suite.
</ol>

There are several different approaches to conformance testing of OSPF. To test an OSPF 
implementation, test cases can be derived using formal methods. However, most test suites
used in practice follow a more intuitive approach. Test suites are developed by reading the
standard and developing the test cases directly from this. In addition, there are some
commercial test devices available, which can be used for conformance testing of real routers. 
For the implementation of an OSPF test suite for SSFNet, all these different approaches must 
be discussed  with respect to the above criteria.<p>

Formal methods provide provable coverage and correctness of the resulting test suite. 
Nevertheless, they are not very common in practical testing. Most test suites for OSPF 
conformance testing are designed by reading the standard and extracting the relevant parts. 
Although the test suites which are developed this way cannot be proven to cover the whole 
specification, they provide a sufficient coverage of the standard as well. Especially those 
test suites which are used by many vendors provide a good coverage, because they are used 
extensively. So the probability is high, that missing or incorrect tests are found soon.<p> 

To be able to package and distribute a the test suite with SSFNet, no external devices or 
additional software can be used. Since these systems are explicitly designed to test real 
devices, it would require some effort to integrate them into SSFNet -- if this is possible 
at all. Another negative aspect of these commercial products is their price.<p> 

Most vendors put their products into lab testing at some time. One popular testing lab is
the <a href="http://www.iol.unh.edu">InterOperability Lab</a> at University of New 
Hampshire. The IOL has developed various test suites for different protocols. One of these
protocols is OSPF.<p>

The IOL test suite description is publicly available. All the tests of the 
IOL test suite consist of a test scenario description together with a test procedure. The 
test purposes and test procedures of the test cases are intuitively clear. DML configurations 
may be directly obtained from the scenario descriptions. That makes it easy to implement these 
tests in SSFNet. Since the single test cases are very simple, they can be used for testing 
during development as well. However, there are some features, that are not covered by the 
IOL test suite. There are only tests for broadcast networks -- tests specific to point-to-point 
networks are not taken into account. For most of the test cases, this does not play a role, 
since the network type only affects the addressing of packets and the decision, when to build 
an adjacency. The tests for exactly these features have to be repeated, when broadcast 
networks are implemented. For the remaining tests, broadcast networks can be replaced by a 
set of point-to-point links (see Figure 2). Other tests, missing from the IOL test suite are 
tests for the simple calculation of intra area routes, using the Dijkstra algorithm. However, 
appropriate tests can be added easily to complete the test suite.<p>

<div align="center"><img src="pictures/broadcast_ptp.jpg" width=350 height=99><br>&nbsp;<br>
<b>Figure 2</b> - Replacing broadcast networks</div><p>

Although some tests are missing from the IOL test suite, it is a good basis for practical 
testing early in the development. It can be easily implemented in the SSFNet environment. 
Many vendors use the IOL test service -- and with this the IOL test suite -- when testing 
their products. This implies, that the IOL test suite provides a reasonable coverage and 
correctness for conformance testing. These considerations led to the decision to use the 
IOL test suite as a basis for testing the OSPF implementation in SSFNet. Whenever necessary, 
additions or modifications are made to meet the specific requirements in the SSFNet 
environment.<p>

<br><hr>
<div align="center"> 
<a href="http://ssfnet.d-jacob.net">Home</a> | 
<a href="index.html">Up</a> | 
<a href="ospfv2.html">Previous</a> | 
<a href="design.html">Next</a>
</div>
<hr><br>

<i>Last update: 2002-02-05, Dirk Jacob (<a href="mailto:dirk@d-jacob.net">dirk@d-jacob.net</a>)</i>
</body>
</html>